Countable state Markov processes: non-explosiveness and moment function
نویسنده
چکیده
The existence of a moment function satisfying a drift function condition is well-known to guarantee non-explosiveness of the associated minimal Markov process (cf.[1, 6]), under standard technical conditions. Surprisingly, the reverse is true as well for a countable space Markov process. We prove this result by showing that recurrence of an associated jump process, that we call the α-jump process, is equivalent to non-explosiveness. Non-explosiveness corresponds in a natural way to the validity of the Kolmogorov integral relation for the function identically equal to 1. In particular, we show that the α-jump chain is positive recurrent, all bounded functions satisfy the Kolmogorov integral relation. Positive recurrence can be characterised by a drift function criterion as well. If to a drift function V , there corresponds another drift function W , which is a moment with respect to V , then via a transformation argument, the above relations hold for the transformed process with respect to V . Transferring the results back to the original process, allows to characterise which V -bounded functions satisfy the Kolmogorov forward equation.
منابع مشابه
Countable State Markov Decision Processes with Unbounded Jump Rates and Discounted Cost: Optimality Equation and Approximations
This paper considers Markov decision processes (MDPs) with unbounded rates, as a function of state. We are especially interested in studying structural properties of optimal policies and the value function. A common method to derive such properties is by value iteration applied to the uniformised MDP. However, due to the unboundedness of the rates, uniformisation is not possible, and so value i...
متن کاملThe Rate of Rényi Entropy for Irreducible Markov Chains
In this paper, we obtain the Rényi entropy rate for irreducible-aperiodic Markov chains with countable state space, using the theory of countable nonnegative matrices. We also obtain the bound for the rate of Rényi entropy of an irreducible Markov chain. Finally, we show that the bound for the Rényi entropy rate is the Shannon entropy rate.
متن کاملOn the Poisson Equation for Markov Chains : Existence of Solutions and Parameter Dependenceby Probabilistic
This paper considers the Poisson equation associated with time-homogeneous Markov chains on a countable state space. The discussion emphasizes probabilistic arguments and focuses on three separate issues, namely (i) the existence and uniqueness of solutions to the Poisson equation, (ii) growth estimates and bounds on these solutions and (iii) their parametric dependence. Answers to these questi...
متن کاملBULLETIN (New Series) OF THE AMERICAN MATHEMATICAL SOCIETY
A random walk, a synonym in a loose sense for a time homogeneous Markov chain on a countable state space, used to be an object that probabilists were handling as a typical stochastic process. To be more precise, given a countable set V on which a walker moves around, we consider a probability law under which the walker transfers from one site to another. This law is described by a non-negative ...
متن کاملInfinite Excess Entropy Processes with Countable-State Generators
We present two examples of finite-alphabet, infinite excess entropy processes generated by invariant hidden Markov models (HMMs) with countable state sets. The first, simpler example is not ergodic, but the second is. It appears these are the first constructions of processes of this type. Previous examples of infinite excess entropy processes over finite alphabets admit only invariant HMM prese...
متن کامل